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Docket No. 18525-0802 

METHOD AND SYSTEM FOR IMAGE PROCESSING 

This application claims priority of U.S. Provisional Patent Application Serial No.: 
60/436,339, filed December 24, 2002. 

TECHNICAL FIELD 

The present invention relates to the automated processing of images, in particular to 
the scanning and processing of images of mail pieces to decode address and other 
information. 

BACKGROUND OF THE INVENTION 

Current state-of-the art postal letter mail equipment is divided into two types, those 
that primarily process addresses, and those that process mail that has been bar coded. Bar 
code processing equipment is simpler in design and does not require as long a transport belt 
path because the time to process a mail piece image is less for one containing a bar code 
than one containing only address information. Bar codes typically included on mail items 
are traditional linear bar code types as well as two dimensional code patterns that contain 
alphanumeric and ZIP code data. 

Many efforts to improve bar code recognition have been employed to increase the 
effective yield of the machines to as high as 90%, but ultimately severely degraded images, 
or those wherein the bar code is obscured by an envelope's window edge, cause a certain 
percentage of mail items to be rejected. Mail items with bar codes that cannot be read are 
rejected and either processed manually or reprocessed on other equipment capable of 
reading the full address. These later steps result in additional costs and delay in processing 
the mail items, and depending upon the specific step of the production process, this can 
sometimes result in an extra day to deliver the item. 

Some bar-coded mail items have an additional identification code used to associate 
an image with the mail item. These pieces had been previously processed and a sorting bar 
code applied by an OCR sorting system as deployed by the USPS, or by a mail processing 
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companies that have equipment similar to the USPS and who are operating under Hcense to 
the USPS. The ID code is read by a separate bar code reader and is used by the bar code 
sorters to enable each to query a central server for bar code equivalent data when sorting 
bar codes cannot be read. The server receives the bar code equivalent data, i.e., data from 
5 which a correct destination bar code for that mail piece can be determined, from a process 
such as video coding. A need persists for a system able to provide a higher read success 
rate for bar code sorters. 

SUMMARY OF THE INVENTION 

10 The invention provides an improved method and system for processing mail items 

on bar code sorting machines. Such a method includes the steps of: 

(a) scanning a surface of each mail piece with at least one imaging camera to obtain 
at least one image of the surface represented by image data; 

(b) analyzing the image data to locate a bar code in the image; 
15 (c) analyzing the bar code to recognize a first destination code; 

(d) if the first destination code meets predetermined criteria, then processing the 
mail piece further in a manner dependent on the first destination code; and 

(e) if the first destination code cannot be recognized or fails to meet the 
predetermined criteria, then analyzing the image data to locate destination address lines in 

20 the image, analyzing the address lines to determine a second destination code, and 

processing the mail piece further in a manner dependent on one of the second destination 
code, or a result determined by arbitrating the first and second destination codes. 

These steps are generally executed on a mail sorting machine which may be 
equipped with one or two imaging cameras or scanners. If two cameras are present, then 

25 step (a) contemplates that one image can be made to locate the bar code and another to 
locate destination address lines. In the alternative, a single image can be analyzed using 
known region of interest strategies to obtain both the destination address lines image data 
and the bar code image data. The process of arbitration, described further below, is 
optional. In the simplest embodiment of the invention, if the bar code does not adequately 

30 give the destination code (unreadable, outdated, or only gives part of the code), then the 

second destination code is used to determine how the item is sorted or otherwise processed. 



If both the bar code and address lines prove unreadable, the mail item is a reject. 

The invention further provides an apparatus for sorting a series of mail pieces 
according to the method set forth above. Such an apparatus includes a conveyor which 
transports a series of mail pieces, generally a pair of upright belts holding each mail piece 
from either side for transport on edge. The conveyor ultimately conveys each mail piece to 
a series of diverters which are each operable to divert mail pieces from the conveyor to one 
of a series of sorting destinations, e.g., bins or slots. An imaging camera is positioned 
proximate the conveyor for scanning a surface of each mail piece to obtain an image of the 
surface represented by image data. A computerized control system analyzes the image data 
to locate a bar code in the image and analyzes the bar code to recognize a first destination 
code. If the first destination code meets predetermined criteria, the control system sorts the 
mail piece by actuating one of the diverters in a manner dependent on the first destination 
code. If the first destination code cannot be recognized or fails to meet the predetermined 
criteria, the system then analyzes the image data to locate destination address lines in the 
image, analyzes the address lines to determine a second destination code, and sorts the mail 
piece by actuating one of the diverters in a manner dependent on one of the second 
destination code, or a result determined by arbitrating the first and second destination 
codes. These steps can be carried out by any appropriate combination of hardware and 
software. 

These and other aspects of the invention are further described and discussed in the 
detailed description which follows. 

BRIEF DESCRIPTION OF THE DRAWING 
In the accompanying drawing, wherein like numerals represent the same or similar 

elements throughout: 

Figure 1 is a typical sorting machine elevator section which can be used in the 

invention; 

Figure 2 is a schematic diagram of a system for scanning mail pieces according to 
the invention; and 

Figure 3 is a schematic diagram of an alternative apparatus according to the 
invention. 
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DETAILED DESCRIPTION 

Optical character reader (OCR)-assisted bar code reading (BCR) according to the 
invention can provide recognition of the address and other text and pattern information on 
the mail piece for subsequent sortation to the correct machine pocket. Significant 
5 performance improvements on the remaining bar code rejects can be gained utilizing this 
method. A computer processor is used to implement the OCR and pattern recognition on 
the traditional bar code sorting machines (a total of over 8,000 USPS bar code sorters, 
including Delivery Point Bar Code Sorters-DBCS, Carrier Sequencing Bar Code Sorters- 
CSBCS and Delivery Input Output Sub Systems-DIOSS). The existing sorter system 

10 computer is augmented with an OCR/pattern recognition computer. This computer 
performs the traditional bar code reading function. OCR and text pattern recognition 
software processes are initiated for those mail pieces that have no bar code (i.e. bar code 
slipped behind the window) or have an unreadable bar code, thereby providing additional 
read performance for mail items that do not contain a bar code or contain unreadable or 

1 5 damaged bar codes. 

Referring to Figures 1 and 2, a system 10 according to the invention includes a 
paired belt conveyor 1 1 which is part of a postal sorting machine. At the sorting machine 
elevator section shown in Figure 1, mail carried by conveyor 1 1 passes by a wide field of 
view digital camera 13. Camera 13 captures an image of the mail piece surface that will be 

20 analyzed for address information in accordance with the invention. Immediately after 

scanning, each mail piece passes by a first photo cell or light barrier 14 and enters an idler 
section 16 of the conveyor that leads to the diverter gates 17. In typical bar code sorting 
machines used in the U.S. postal system, first photo cell 14 is located some distance from 
the first mail path diverter gate 17 that separates mail streams into different levels. In this 

25 implementation, the time from photo cell 14 to a second diverter gate photo cell 18 can be 
utilized for arbitration of differing results and returning the best ZIP code and sort decision 
result. In this implementation, in other words, the additional latency time is utilized for 
processing. This would not be possible with a standard configuration that requires the bar 
code reader output results at a fixed time interval. 

30 The output of the image lift camera 13 is passed to both bar code recognition 21 and 

OCR recognition 22 process modules which form part of the control system 20. Output 
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from photocell 14 is provided to image lift camera 13 for transfer to the bar code 
recognition module 21 for synchronization with the bar code recognition process and also 
to the OCR recognition module 22 to assure that the OCR process is synchronized with 
results that are determined by the bar code recognition process. Numerous methods of can 
5 be employed for synchronizing these data such as time stamping the output, optical encoder 
counts, or other methods that are well known in the art. Both bar code and OCR processes 
21, 22 can be initiated in parallel with the acquisition of the image data. Alternatively, 
either process can await suitable portions of the image information to be delivered prior to 
beginning process execution. Processes 21, 22 are embodied in program logic that can be 
10 run as multiple tasks on a single computer, or multiple computers or processors may be 
provided. 

Bar code recognition normally completes execution sooner than OCR recognition. 
The results of this process are provided to the machine control module 24, operable from a 
user terminal 23, which determines the destination sort location for the item represented by 

15 the image. This result is buffered to await the OCR process resolution, or an interrupt from 
the sort decision second photocell 18 that informs machine control 24 that a decision must 
be provided. Occasionally, the result from the bar code decision process 21 is that there is 
no valid result and subsequent sort decision. 

"Buffering" as used in the present invention refers to the process of maintaining in 

20 computer memory or storing to a computer storage medium data a series of data elements, 
such as images captured from a device such as a camera or scanner. Generally the buffer 
will contain such data long enough for an intended purpose, such as decoding, to be 
completed. At typical sorting speeds, 10 or so mail pieces may be in the process of being 
transported between photocells 14 and 18. The computer memory buffer maintains image 

25 data for these mail pieces in order of arrival, and the processes described are carried on 

simultaneously for each of the mail pieces then in transit. Data is dropped from the buffer, 
if not saved for later reference, once the associated mail piece passes by the second 
photocell 18. 

When the OCR decision data are received from the OCR process, the data are 
30 compared with the bar code decision data for the same mail piece. An arbitration process 
25 determines which result to use and provides that output to machine control. Machine 
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control 24 can also provide a result to OCR process 22 to inform it that a successful bar 
code result has been received. Depending upon the system resources then available, the 
OCR module 22 can terminate processing on the appropriate image, or continue to 
completion. If resources for processing a subsequent image are marginal or low, this 
5 feature enables stopping processing on one image to reallocate resources (memory or 
machine cycles) to work on a subsequent one. 

In some systems, such as DIOSS machines, the machine is also provided with an 
bar code ID tag or RFID tag reader 26. These devices permit the machine to read ID codes 
off of a mail piece previously determined to be unreadable in a postal sorting process. The 

10 mail piece was ID tagged and diverted for human review (video coding) so that a valid 
address and destination code could be determined. When such a tag is detected, control 
module 24 consults an external database wherein the results of video coding are correlated 
with ID codes. The destination ZIP code to be used for sorting or other processing is then 
the one matching the ID tag, which would override any result determined by OCR or bar 

1 5 code processes 2 1 , 22. 

If idler section 16 is sufficiently long and sufficient computing resources are 
available, it is preferred according to the invention for the results from processes 21, 22 to 
be processed by arbitrator module 25. "Arbitration" in postal processing refers generally to 
software (program logic) that compares results from two different, usually parallel 

20 processes and decides which to use based on predetermined criteria or selection rules. 

Arbitration can also involve combining information from two or more upstream processes 
to reach a result most likely to be accurate. In the process of the present invention, 
arbitration is not normally needed if, as in a majority of cases, the destination bar code is 
fully readable (to eleven digits in the case of a ZIP code.) Arbitration may be needed where 

25 the results from processes 21, 22 return a different number of digits, for example, the bar 
code is readable to 5 or 7 digits but the OCR process results in 1 1 digits. If the first 5 or 7 
digits of the OCR process match the bar code result, the arbitration rules might then dictate 
that the 1 1 digit result be used. Arbitration can also be used in the event that the bar code is 
readable but proves to be invalid, outdated or incorrect to minimize sorting on such bar 

30 codes. The result of arbitration is sent to control module 24 and is used as the basis for 
further processing. The rules for arbitration are determined in advance according to user 
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preference. 

As noted above, the bar code recognition process 21 and OCR process 22 may run 
in parallel. For example, OCR process 22 works on locating the address coordinates and 
location of the address lines while bar code process 21 decodes the bar code. If bar code 
5 recognition is not successful, or if the bar code does not meet predetermined criteria, then 
process 22 continues to determine the destination ZIP Code and other address data. In the 
alternative, time permitting, the start of process 22 is delayed until process 21 is completed 
and a bar code that does not meet the conditions for validity (e.g., unreadable, not enough 
digits, obsolete or invalid) has been detected. Upon completion of process 22, the 

10 destination address is converted into bar code equivalent data, that is, a delivery code such 
as a ZIP code. This may require reference to a master database of addresses and associated 
delivery codes, as in a case where the address can be read but no zip code is provided or 
only a 5-digit zip code is provided. 

According to another aspect of the invention, an ID tag may be applied to mail 

15 pieces for which processes 21 and 22 fail to result in a destination code enabling a sorting 
decision. This is the starting end of the process discussed above wherein previously 
applied ID or radiofrequency ID tags are read. As shown in Figure 3, in an alternative 
embodiment of the invention, a DIOSS machine 30 includes an ID tagger 31 downstream 
from camera 13, preferably a sufficient distance so that a decision can be made concerning 

20 each mail piece before it passes, tagger 31. An ink jet printer 32 then prints the scannable 
ID code on to the mail piece, and the mail piece enters a drying loop 33 before it reaches 
the first diverter gate 17. The control computer stores the image data indexed to the ID 
number in a storage medium (e.g., hard drive) for later retrieval. The OCR results, such as 
a partial address, may also be stored for later review, indexed to the same ID number. The 

25 mail piece is then sorted to a reject bin for subsequent processing, such as video coding. 
Once video coded, the mail piece can then be reintroduced to a sorter equipped with an ID 
tag reader as described above. 

Use of the systems and processes of the invention can improve read rates on 
currently deployed bar code sorters by from 1 to 3 percent, resulting in large cost savings 

30 without any significant changes to the postal processing equipment. Although various 
embodiments of the invention have been illustrated in the accompanying drawings and 
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described in the foregoing detailed description, it will be understood that the invention is 
not limited to the embodiments disclosed but, as will be appreciated by those skilled in the 
art, is susceptible to numerous modifications and variations without departing from the 
spirit and scope of the invention as hereinafter claimed. 



